SoftBotSearch : SoftBot Approach to information retrieval from Web

In this paper, we describe SoftBotSearch, an Internet softbot which organizes information retrieved from search engines in a form that is more useful to humans. SoftBotSearch meta-searches the web using three popular search engines, retrieves the web pages from the Internet and groups them according to their theme. Thus a search for “operating system” automatically organizes the results into categories like Microsoft, open source resources, distributed operating systems, memory management etc. We use the K-Means and Subtractive Clustering algorithms to find clusters in the document vector space. In this paper we also define methods used for clustering including suffix tree clustering algorithm used. SoftBotSearch can be easily adapted to work on any information source.